The present invention relates generally to computer software, and more particularly, to a system and method for a scalable, high performance computing infrastructure for distributed software applications over a computer network.
With the rapid expansion of both local and wide area computer networks, it is no longer an unreachable goal to link geographically distributed network sites (or nodes) together. This has led to a demand for the development of distributed software applications that take advantage of the revolutionized computing connectivity. A distributed computer system will have various software applications located on different local nodes, but are concerted to act together. Writing reliable distributed software applications for a distributed computer system has been a difficult and error-prone activity due to a variety of reasons such as asynchrony of different nodes in the distributed network, individual node or network failures, network partitioning, or uncontrollable network latencies, etc. Although each local distributed software application may be functional at the local node, maintaining a consistent global state has been enormously difficult. In the face of this complexity, programmers are forced to create centralized solutions that are rarely reusable, and have poor fault tolerance. Therefore, actual implementations of a fault tolerant, high performance distributed computer system hardly exist in the industry.
The design as well as the verification of a fault-tolerant distributed system is a highly non-trivial task and rests on a few fundamental paradigms. One key paradigm among these is consensus. Consensus can be viewed as a general form of agreement in the distributed computer system, wherein a group of processes reach a common decision based on their initial inputs, despite failures of some of the processes in the group. Atomic Broadcast, another important paradigm in the distributed systems, has been shown to be equivalent to consensus. Further, many frequently occurring problems such as Atomic Commitment, Group Membership and View Synchrony can be reduced to an appropriate consensus based problem. Besides the consensus issue, reliability of such a distributed computer system is also a key concern. Various applications for a distributed computer system that require a high reliability include air traffic control systems, automation software for industries, financial and brokerage software and many more.
A method and system is thus needed for dealing with all consensus based problems in a high performance distributed computer system that is tolerant to node or network failures and latencies.
A system and method is provided for a high performance distributed computer network system that is tolerant to node or network failures and latencies. The distributed computer system is a dynamic clustering computing system where distributed objects can join and leave a cluster as an application demands. It effectively deals with all consensus based problems in the distributed computer network having various consensus based application protocols.
In some embodiments, the system is a common platform supporting heterogeneous operating systems and provides structuring mechanisms to present a global state of a distributed application using an xe2x80x9cobjects as groupsxe2x80x9d paradigm.
In some embodiments, structuring abstraction elements such as transactional messaging and group communications are available in a distributed computer network. These embodiments thus provide a software-based approach to object clustering where distributed objects are the units for clustering together and an object group abstraction is the principal structuring element.
Some embodiments include an event bus for transporting all communication messages among objects or processes in the distributed computer system. These embodiments also have a fault-tolerant, client-server architecture where a client process presents a particular consensus based problem to server processes to start a consensus service in order to solve such a consensus based problem. In addition, one or more priority failure detectors may be included that detect process failures in the distributed computer network. The present invention assigns priority levels to processes involved in a consensus session and controls the frequencies of their heartbeat status messages based on their respective priority levels. By controlling the frequencies, the reliability of the network is enhanced because processes having high priority can have a high rate of message communication. On the other hand, the overall message load on the event bus can be controlled and reduced to a minimum number due to a reduction of status messages for the low priority processes.
Some embodiments also disclose a Name Service that assigns unique logical identities to all processes in a consensus service. In one embodiment of the present system, during the initiation of a consensus service, all participating consensus server processes are required to register with the Name Service, and logical identities will be assigned only after all participating consensus server processes complete the registration process.
In another embodiment, the present invention handles multiple consensus based problems concurrently. By tagging all involved processes appropriately, multiple consensus based problems can be dealt with on a set of consensus server processes simultaneously.
The present invention thus improves the performance and enhances the reliability of such a distributed computer network.